1. Field of Invention
The present invention pertains to the field of classifiers. More particularly, this invention relates to determining accuracy of a classifier.
2. Art Background
A classifier may be defined as an entity that associates an item with one or more of a set of categories. Typically, a classifier determines which categories with which to associate an item in response to attributes of the item. Examples of classifiers are numerous and include rule-based systems, neural networks, Bayesian probability systems, as well as human beings.
It is commonly desirable to determine the accuracy of a classifier. For example, the accuracy of a classifier may be used to improve the classifier or to compare the relative accuracy of different classifiers.
One prior method for determining the accuracy of a classifier is to compare classifications rendered by the classifier with classifications rendered by an expert. Such a method usually yields a boolean correct/incorrect indication of accuracy with respect to the classification of individual items. The boolean indications may be aggregated to obtain a measure of the accuracy of the classifier. Unfortunately, such correct/incorrect indications are usually of relatively limited utility in evaluating the relative performance of a classifier. Moreover, such indications usually do not provide information on the extent to which an individual classification is wrong.